Three-dimensional visualization environment for multisensor data analysis, interpretation, and model-based object recognition

Model-based object recognition must solve three-dimensional geometric problems involving the registration of multiple sensors and the spatial relationship of a three-dimensional model to the sensors. Observation and verification of the registration and recognition processes requires display of these geometric relationships. We have developed a prototype software system which allows a user to interact with the sensor data and model matching system in a three-dimensional environment. This visualization environment combines range imagery, color imagery, thermal (infrared) imagery, and CAD models of objects to be recognized. We are currently using imagery of vehicles travelling off-road (a challenging environment for the object recognizer). Range imagery is used to create a partial three-dimensional representation of a scene. Optical imagery is mapped onto this partial 3D representation. Visualization allows monitoring of the recognizer as it solves for the type and position of the object. The object is rendered from its associated CAD model. In addition to its usefulness in development of the object recognizer, we foresee eventual use of this technology in a fielded system for operator verification of automatic target recognition results.